Learning Compositional Visual Concepts with Mutual Consistency

Compositionality of semantic concepts in image synthesis and analysis is appealing as it can help in decomposing known and generatively recomposing unknown data. For instance, we may learn concepts of changing illumination, geometry or albedo of a scene, and try to recombine them to generate physically meaningful, but unseen data for training and testing. In practice however we often do not have samples from the joint concept space available: We may have data on illumination change in one data set and on geometric change in another one without complete overlap. We pose the following question: How can we learn two or more concepts jointly from different data sets with mutual consistency where we do not have samples from the full joint space? We present a novel answer in this paper based on cyclic consistency over multiple concepts, represented individually by generative adversarial networks (GANs). Our method, ConceptGAN, can be understood as a drop in for data augmentation to improve resilience for real world applications. Qualitative and quantitative evaluations demonstrate its efficacy in generating semantically meaningful images, as well as one shot face verification as an example application.Comment: 10 pages, 8 figures, 4 tables, CVPR 201

Tensor Factorization for Leveraging Cross-Modal Knowledge in Data-Constrained Infrared Object Detection

The primary bottleneck towards obtaining good recognition performance in IR images is the lack of sufficient labeled training data, owing to the cost of acquiring such data. Realizing that object detection methods for the RGB modality are quite robust (at least for some commonplace classes, like person, car, etc.), thanks to the giant training sets that exist, in this work we seek to leverage cues from the RGB modality to scale object detectors to the IR modality, while preserving model performance in the RGB modality. At the core of our method, is a novel tensor decomposition method called TensorFact which splits the convolution kernels of a layer of a Convolutional Neural Network (CNN) into low-rank factor matrices, with fewer parameters than the original CNN. We first pretrain these factor matrices on the RGB modality, for which plenty of training data are assumed to exist and then augment only a few trainable parameters for training on the IR modality to avoid over-fitting, while encouraging them to capture complementary cues from those trained only on the RGB modality. We validate our approach empirically by first assessing how well our TensorFact decomposed network performs at the task of detecting objects in RGB images vis-a-vis the original network and then look at how well it adapts to IR images of the FLIR ADAS v1 dataset. For the latter, we train models under scenarios that pose challenges stemming from data paucity. From the experiments, we observe that: (i) TensorFact shows performance gains on RGB images; (ii) further, this pre-trained model, when fine-tuned, outperforms a standard state-of-the-art object detector on the FLIR ADAS v1 dataset by about 4% in terms of mAP 50 score.Comment: Accepted to ICCV 2023, LIMIT Workshop. The first two authors contributed equall

Are Deep Neural Networks SMARTer than Second Graders?

Recent times have witnessed an increasing number of applications of deep neural networks towards solving tasks that require superior cognitive abilities, e.g., playing Go, generating art, ChatGPT, etc. Such a dramatic progress raises the question: how generalizable are neural networks in solving problems that demand broad skills? To answer this question, we propose SMART: a Simple Multimodal Algorithmic Reasoning Task and the associated SMART-101 dataset, for evaluating the abstraction, deduction, and generalization abilities of neural networks in solving visuo-linguistic puzzles designed specifically for children in the 6--8 age group. Our dataset consists of 101 unique puzzles; each puzzle comprises a picture and a question, and their solution needs a mix of several elementary skills, including arithmetic, algebra, and spatial reasoning, among others. To scale our dataset towards training deep neural networks, we programmatically generate entirely new instances for each puzzle, while retaining their solution algorithm. To benchmark performances on SMART-101, we propose a vision and language meta-learning model using varied state-of-the-art backbones. Our experiments reveal that while powerful deep models offer reasonable performances on puzzles in a supervised setting, they are not better than random accuracy when analyzed for generalization. We also evaluate the recent ChatGPT and other large language models on a part of SMART-101 and find that while these models show convincing reasoning abilities, the answers are often incorrect.Comment: Accepted at CVPR 2023. For the SMART-101 dataset, see https://doi.org/10.5281/zenodo.776179

Enhanced Differentiation of Three-Gene-Reprogrammed Induced Pluripotent Stem Cells into Adipocytes via Adenoviral-Mediated PGC-1α Overexpression

Induced pluripotent stem cells formed by the introduction of only three factors, Oct4/Sox2/Klf4 (3-gene iPSCs), may provide a safer option for stem cell-based therapy than iPSCs conventionally introduced with four-gene iPSCs. Peroxisome proliferator-activated receptor gamma coactivator-1α (PGC-1α) plays an important role during brown fat development. However, the potential roles of PGC-1α in regulating mitochondrial biogenesis and the differentiation of iPSCs are still unclear. Here, we investigated the effects of adenovirus-mediated PGC-1α overexpression in 3-gene iPSCs. PGC-1α overexpression resulted in increased mitochondrial mass, reactive oxygen species production, and oxygen consumption. Microarray-based bioinformatics showed that the gene expression pattern of PGC-1α-overexpressing 3-gene iPSCs resembled the expression pattern observed in adipocytes. Furthermore, PGC-1α overexpression enhanced adipogenic differentiation and the expression of several brown fat markers, including uncoupling protein-1, cytochrome C, and nuclear respiratory factor-1, whereas it inhibited the expression of the white fat marker uncoupling protein-2. Furthermore, PGC-1α overexpression significantly suppressed osteogenic differentiation. These data demonstrate that PGC-1α directs the differentiation of 3-gene iPSCs into adipocyte-like cells with features of brown fat cells. This may provide a therapeutic strategy for the treatment of mitochondrial disorders and obesity

Women with endometriosis have higher comorbidities: Analysis of domestic data in Taiwan

AbstractEndometriosis, defined by the presence of viable extrauterine endometrial glands and stroma, can grow or bleed cyclically, and possesses characteristics including a destructive, invasive, and metastatic nature. Since endometriosis may result in pelvic inflammation, adhesion, chronic pain, and infertility, and can progress to biologically malignant tumors, it is a long-term major health issue in women of reproductive age. In this review, we analyze the Taiwan domestic research addressing associations between endometriosis and other diseases. Concerning malignant tumors, we identified four studies on the links between endometriosis and ovarian cancer, one on breast cancer, two on endometrial cancer, one on colorectal cancer, and one on other malignancies, as well as one on associations between endometriosis and irritable bowel syndrome, one on links with migraine headache, three on links with pelvic inflammatory diseases, four on links with infertility, four on links with obesity, four on links with chronic liver disease, four on links with rheumatoid arthritis, four on links with chronic renal disease, five on links with diabetes mellitus, and five on links with cardiovascular diseases (hypertension, hyperlipidemia, etc.). The data available to date support that women with endometriosis might be at risk of some chronic illnesses and certain malignancies, although we consider the evidence for some comorbidities to be of low quality, for example, the association between colon cancer and adenomyosis/endometriosis. We still believe that the risk of comorbidity might be higher in women with endometriosis than that we supposed before. More research is needed to determine whether women with endometriosis are really at risk of these comorbidities